{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Intro\n",
    "\n",
    "This is inspired by \n",
    "Article (likas2001probability) Likas, A. Probability density estimation using artificial neural networks Computer physics communications, Elsevier, 2001, 135, 167-175\n",
    "\n",
    "But rather than estimating the working with a network, we will instead work with its derivitive.\n",
    "This will let us replace their integration with a derivative.\n",
    "\n",
    "Note that this method only works for compact supports\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "They use the PDF is given by $$p_h(x,p) = \\dfrac{h(x,p)}{\\int_S h(z,p) dz}$$\n",
    "and in their case $h=N(x,p)$  a neural network with weight and bias parameters $p$.\n",
    "Where $S$ is a compact support. (That means bounded)\n",
    "\n",
    "\n",
    "But if instead we say $h=\\frac{\\partial N(x,p)}{\\partial x}$,\n",
    "\n",
    "then $$p_h(x,p) = \\dfrac{h(x,p)}{\\int_S h(z,p)}=\\dfrac{\\frac{\\partial N(x,p)}{\\partial x}}{N(max(S),p) - N(min(S), p)}$$\n",
    "\n",
    "The denominator is of-course more complex for non-1D values of S.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The loss function given is the negative log-likelihood of the set of training samples $X$\n",
    "$$L(p) = -\\sum_{\\forall x \\in X} ln(h(x,p))  + |X| ln(\\int_S h(z,p) dx)$$\n",
    "\n",
    "Which befomes:\n",
    "\n",
    "$$L(p) = -\\sum_{\\forall x \\in X} log(\\frac{\\partial N(x,p)}{\\partial x})  + |X|(ln(N(max(S),p)-N(min(S),p)) dx$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "only (generic function with 1 method)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using DensityEstimationML\n",
    "function only(itr)\n",
    "    state = start(itr)\n",
    "    val,state = next(itr, state)\n",
    "    @assert(done(itr,state))\n",
    "    return val\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "using TensorFlow\n",
    "using Distributions\n",
    "using StatsBase\n",
    "using StaticArrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "running_plot"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using Plots\n",
    "using IJulia\n",
    "\n",
    "\"\"\"\n",
    "Function returning a function that will display a running plot.\n",
    "WARNING: Introducting or removing any variables is not supported.\n",
    "And will silently error.\n",
    "\"\"\"\n",
    "function running_plot()\n",
    "    epochs = Int[]\n",
    "    record = Dict()\n",
    "    function inner(epoch, vars::Associative)\n",
    "        for (var, values) in vars\n",
    "            value = only(values) #Incase it was an array\n",
    "            past = get!(record, var) do\n",
    "                typeof(value)[]\n",
    "            end\n",
    "            push!(past, value)\n",
    "        end\n",
    "        push!(epochs, epoch)\n",
    "        \n",
    "        IJulia.clear_output(true)\n",
    "        plot(epochs, hcat(values(record)...); label=hcat(keys(vars)...), layout=length(vars)) |> IJulia.display       \n",
    "    end\n",
    "end\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "demonstration_plot (generic function with 1 method)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function demonstration_plot(est, dataset, data, args...; kwargs...)\n",
    "    X = minimum(approximate_support(dataset)) : 0.01 : maximum(approximate_support(dataset)) |> collect\n",
    "    println(\"True loglikelihood      = $(loglikelihood(dataset, data))\")\n",
    "    println(\"Estimated loglikelihood = $(loglikelihood(est, data))\")\n",
    "    plot([X], [pdf(est,X), data],\n",
    "        #xlims= approximate_support(dataset),\n",
    "        xlims= (first(X), last(X)),\n",
    "        seriestype = [:path :histogram],\n",
    "        layout=(2,1),\n",
    "        legend=false,\n",
    "        nbins=[1  length(data)÷10],\n",
    "        args...; kwargs...\n",
    "    )\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "demo (generic function with 2 methods)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function demo(dataset, layers, epochs=20_000; max_conditioning_epochs=2000)\n",
    "    data = original_sample(dataset)\n",
    "    @show loglikelihood(dataset, data)\n",
    "    est = NeuralDensityEstimator(layers, approximate_support(dataset))\n",
    "\n",
    "    condition!(est; max_epochs = max_conditioning_epochs)\n",
    "    println(\"Conditioning Done\")\n",
    "    Plots.gr()\n",
    "    fit!(est, data; epochs=epochs, callback=running_plot())\n",
    "    println(\"Fitting Done\")\n",
    "    pyplot()\n",
    "    \n",
    "    demonstration_plot(est, dataset, data) |> IJulia.display\n",
    "    \n",
    "    est    \n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"600\" height=\"400\" viewBox=\"0 0 600 400\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip00\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"600\" height=\"400\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "0,400 600,400 600,0 0,0 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip01\">\n",
       "    <rect x=\"120\" y=\"0\" width=\"421\" height=\"400\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "55.6177,175.813 283.776,175.813 283.776,11.811 55.6177,11.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip02\">\n",
       "    <rect x=\"55\" y=\"11\" width=\"229\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  113.21,173.353 113.21,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  170.826,173.353 170.826,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  228.442,173.353 228.442,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,163.797 280.353,163.797 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,104.238 280.353,104.238 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,44.6801 280.353,44.6801 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,175.813 283.776,175.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  113.21,175.813 113.21,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  170.826,175.813 170.826,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  228.442,175.813 228.442,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,175.813 55.6177,11.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,163.797 59.04,163.797 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,104.238 59.04,104.238 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,44.6801 59.04,44.6801 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 113.21, 189.613)\" x=\"113.21\" y=\"189.613\">2500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 170.826, 189.613)\" x=\"170.826\" y=\"189.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 228.442, 189.613)\" x=\"228.442\" y=\"189.613\">7500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 168.297)\" x=\"49.6177\" y=\"168.297\">2.5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 108.738)\" x=\"49.6177\" y=\"108.738\">5.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 49.1801)\" x=\"49.6177\" y=\"49.1801\">7.5</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,175.813 57.9223,36.0019 60.2269,11.811 62.5315,30.1126 64.8362,42.1417 67.1408,57.6317 69.4454,74.9049 71.75,87.4311 74.0547,95.2738 76.3593,102.45 \n",
       "  78.6639,103.348 80.9686,91.4136 83.2732,65.3505 85.5778,34.2991 87.8824,22.1332 90.1871,29.1237 92.4917,38.3778 94.7963,44.4891 97.1009,49.5753 99.4056,57.9571 \n",
       "  101.71,67.0802 104.015,72.3958 106.319,72.8572 108.624,71.728 110.929,70.4906 113.233,69.4598 115.538,68.7112 117.843,68.2245 120.147,67.9603 122.452,67.8841 \n",
       "  124.756,67.9688 127.061,68.1931 129.366,68.5404 131.67,68.994 133.975,69.5369 136.28,70.1367 138.584,70.7441 140.889,71.2975 143.193,71.5124 145.498,70.8026 \n",
       "  147.803,69.5999 150.107,68.2633 152.412,67.029 154.717,66.0419 157.021,65.2309 159.326,64.5717 161.63,64.1203 163.935,63.8515 166.24,63.8604 168.544,65.0844 \n",
       "  170.849,76.7441 173.154,82.3738 175.458,85.2989 177.763,87.1806 180.068,88.5726 182.372,89.7079 184.677,90.7073 186.981,91.628 189.286,92.5066 191.591,93.3657 \n",
       "  193.895,94.2296 196.2,95.1251 198.505,96.013 200.809,96.9233 203.114,97.8667 205.418,98.8308 207.723,99.762 210.028,100.68 212.332,101.574 214.637,102.412 \n",
       "  216.942,103.189 219.246,103.953 221.551,104.647 223.855,105.233 226.16,105.682 228.465,106.001 230.769,106.145 233.074,106.127 235.379,106.045 237.683,105.874 \n",
       "  239.988,105.628 242.292,105.476 244.597,105.442 246.902,105.526 249.206,105.74 251.511,106.051 253.816,106.452 256.12,106.924 258.425,107.436 260.729,107.987 \n",
       "  263.034,108.59 265.339,109.22 267.643,109.917 269.948,110.616 272.253,111.318 274.557,112.019 276.862,112.696 279.166,113.391 281.471,114.045 283.776,114.648 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "171.536,62.931 265.776,62.931 265.776,32.691 171.536,32.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  171.536,62.931 265.776,62.931 265.776,32.691 171.536,32.691 171.536,62.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  177.536,47.811 213.536,47.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 219.536, 52.311)\" x=\"219.536\" y=\"52.311\">ysmax</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "352.157,175.813 580.315,175.813 580.315,11.811 352.157,11.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip03\">\n",
       "    <rect x=\"352\" y=\"11\" width=\"229\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  409.75,173.353 409.75,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  467.365,173.353 467.365,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  524.981,173.353 524.981,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,101.39 576.893,101.39 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,20.912 576.893,20.912 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,175.813 580.315,175.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  409.75,175.813 409.75,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  467.365,175.813 467.365,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  524.981,175.813 524.981,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,175.813 352.157,11.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,101.39 355.579,101.39 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,20.912 355.579,20.912 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 409.75, 189.613)\" x=\"409.75\" y=\"189.613\">2500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 467.365, 189.613)\" x=\"467.365\" y=\"189.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 524.981, 189.613)\" x=\"524.981\" y=\"189.613\">7500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 105.89)\" x=\"346.157\" y=\"105.89\">20000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 25.412)\" x=\"346.157\" y=\"25.412\">30000</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,11.811 354.462,113.227 356.766,135.167 359.071,138.52 361.375,139.264 363.68,139.672 365.985,140.12 368.289,140.686 370.594,141.442 372.899,142.981 \n",
       "  375.203,145.308 377.508,147.863 379.812,150.096 382.117,151.908 384.422,154.723 386.726,157.316 389.031,159.43 391.336,161.232 393.64,162.858 395.945,164.47 \n",
       "  398.249,166.211 400.554,168.38 402.859,169.881 405.163,170.624 407.468,171.101 409.773,171.419 412.077,171.634 414.382,171.781 416.686,171.883 418.991,171.956 \n",
       "  421.296,172.009 423.6,172.05 425.905,172.084 428.21,172.113 430.514,172.138 432.819,172.162 435.123,172.184 437.428,172.205 439.733,172.23 442.037,172.317 \n",
       "  444.342,172.65 446.647,173.112 448.951,173.507 451.256,173.715 453.56,173.821 455.865,173.898 458.17,173.964 460.474,174.022 462.779,174.079 465.084,174.219 \n",
       "  467.388,174.478 469.693,174.595 471.998,174.67 474.302,174.729 476.607,174.781 478.911,174.828 481.216,174.873 483.521,174.915 485.825,174.956 488.13,174.995 \n",
       "  490.435,175.032 492.739,175.067 495.044,175.1 497.348,175.131 499.653,175.16 501.958,175.188 504.262,175.214 506.567,175.238 508.872,175.261 511.176,175.284 \n",
       "  513.481,175.305 515.785,175.327 518.09,175.348 520.395,175.37 522.699,175.391 525.004,175.413 527.309,175.434 529.613,175.455 531.918,175.476 534.222,175.496 \n",
       "  536.527,175.516 538.832,175.535 541.136,175.554 543.441,175.573 545.746,175.59 548.05,175.607 550.355,175.624 552.659,175.64 554.964,175.656 557.269,175.672 \n",
       "  559.573,175.687 561.878,175.702 564.183,175.716 566.487,175.73 568.792,175.744 571.096,175.758 573.401,175.772 575.706,175.785 578.01,175.799 580.315,175.813 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "433.823,62.931 562.315,62.931 562.315,32.691 433.823,32.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  433.823,62.931 562.315,62.931 562.315,32.691 433.823,32.691 433.823,62.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  439.823,47.811 475.823,47.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 481.823, 52.311)\" x=\"481.823\" y=\"52.311\">working_loss</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "55.6177,375.813 283.776,375.813 283.776,211.811 55.6177,211.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip04\">\n",
       "    <rect x=\"55\" y=\"211\" width=\"229\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  113.21,373.353 113.21,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  170.826,373.353 170.826,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  228.442,373.353 228.442,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,366.713 280.353,366.713 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,286.237 280.353,286.237 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,375.813 283.776,375.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  113.21,375.813 113.21,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  170.826,375.813 170.826,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  228.442,375.813 228.442,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,375.813 55.6177,211.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,366.713 59.04,366.713 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,286.237 59.04,286.237 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 113.21, 389.613)\" x=\"113.21\" y=\"389.613\">2500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 170.826, 389.613)\" x=\"170.826\" y=\"389.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 228.442, 389.613)\" x=\"228.442\" y=\"389.613\">7500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 371.213)\" x=\"49.6177\" y=\"371.213\">-30000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 290.737)\" x=\"49.6177\" y=\"290.737\">-20000</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,375.813 57.9223,274.386 60.2269,252.446 62.5315,249.1 64.8362,248.359 67.1408,247.954 69.4454,247.508 71.75,246.943 74.0547,246.186 76.3593,244.648 \n",
       "  78.6639,242.32 80.9686,239.764 83.2732,237.528 85.5778,235.71 87.8824,232.889 90.1871,230.296 92.4917,228.183 94.7963,226.381 97.1009,224.756 99.4056,223.145 \n",
       "  101.71,221.406 104.015,219.237 106.319,217.736 108.624,216.993 110.929,216.516 113.233,216.197 115.538,215.982 117.843,215.835 120.147,215.732 122.452,215.66 \n",
       "  124.756,215.607 127.061,215.565 129.366,215.532 131.67,215.503 133.975,215.478 136.28,215.454 138.584,215.433 140.889,215.411 143.193,215.387 145.498,215.299 \n",
       "  147.803,214.966 150.107,214.504 152.412,214.108 154.717,213.9 157.021,213.794 159.326,213.716 161.63,213.651 163.935,213.592 166.24,213.535 168.544,213.396 \n",
       "  170.849,213.139 173.154,213.023 175.458,212.949 177.763,212.891 180.068,212.839 182.372,212.792 184.677,212.748 186.981,212.705 189.286,212.665 191.591,212.626 \n",
       "  193.895,212.589 196.2,212.554 198.505,212.521 200.809,212.49 203.114,212.461 205.418,212.434 207.723,212.408 210.028,212.384 212.332,212.361 214.637,212.339 \n",
       "  216.942,212.317 219.246,212.296 221.551,212.275 223.855,212.253 226.16,212.232 228.465,212.21 230.769,212.189 233.074,212.168 235.379,212.147 237.683,212.127 \n",
       "  239.988,212.107 242.292,212.087 244.597,212.069 246.902,212.05 249.206,212.033 251.511,212.016 253.816,211.999 256.12,211.983 258.425,211.967 260.729,211.952 \n",
       "  263.034,211.937 265.339,211.922 267.643,211.907 269.948,211.893 272.253,211.879 274.557,211.866 276.862,211.852 279.166,211.839 281.471,211.825 283.776,211.811 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "140.567,262.931 265.776,262.931 265.776,232.691 140.567,232.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  140.567,262.931 265.776,262.931 265.776,232.691 140.567,232.691 140.567,262.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  146.567,247.811 182.567,247.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 188.567, 252.311)\" x=\"188.567\" y=\"252.311\">loglikelihood</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "352.157,375.813 580.315,375.813 580.315,211.811 352.157,211.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip05\">\n",
       "    <rect x=\"352\" y=\"211\" width=\"229\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  409.75,373.353 409.75,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  467.365,373.353 467.365,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  524.981,373.353 524.981,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,321.924 576.893,321.924 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,267.688 576.893,267.688 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,213.452 576.893,213.452 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,375.813 580.315,375.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  409.75,375.813 409.75,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  467.365,375.813 467.365,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  524.981,375.813 524.981,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,375.813 352.157,211.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,321.924 355.579,321.924 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,267.688 355.579,267.688 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,213.452 355.579,213.452 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 409.75, 389.613)\" x=\"409.75\" y=\"389.613\">2500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 467.365, 389.613)\" x=\"467.365\" y=\"389.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 524.981, 389.613)\" x=\"524.981\" y=\"389.613\">7500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 326.424)\" x=\"346.157\" y=\"326.424\">2</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 272.188)\" x=\"346.157\" y=\"272.188\">3</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 217.952)\" x=\"346.157\" y=\"217.952\">4</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,372.006 354.462,289.994 356.766,240.536 359.071,222.478 361.375,214.255 363.68,211.811 365.985,211.915 368.289,212.293 370.594,218.841 372.899,235.035 \n",
       "  375.203,249.222 377.508,258.156 379.812,261.456 382.117,260.095 384.422,271.969 386.726,292.036 389.031,308.258 391.336,318.945 393.64,327.31 395.945,336.555 \n",
       "  398.249,344.923 400.554,350.067 402.859,352.112 405.163,352.946 407.468,353.476 409.773,353.913 412.077,354.325 414.382,354.725 416.686,355.113 418.991,355.491 \n",
       "  421.296,355.858 423.6,356.213 425.905,356.554 428.21,356.879 430.514,357.18 432.819,357.451 435.123,357.68 437.428,357.856 439.733,357.935 442.037,357.919 \n",
       "  444.342,358.045 446.647,358.126 448.951,358.034 451.256,357.791 453.56,357.416 455.865,356.925 458.17,356.325 460.474,355.622 462.779,354.933 465.084,355.551 \n",
       "  467.388,362.12 469.693,364.906 471.998,366.272 474.302,367.134 476.607,367.762 478.911,368.264 481.216,368.69 483.521,369.067 485.825,369.411 488.13,369.73 \n",
       "  490.435,370.03 492.739,370.315 495.044,370.589 497.348,370.854 499.653,371.111 501.958,371.361 504.262,371.606 506.567,371.848 508.872,372.088 511.176,372.326 \n",
       "  513.481,372.567 515.785,372.812 518.09,373.061 520.395,373.31 522.699,373.557 525.004,373.797 527.309,374.022 529.613,374.226 531.918,374.407 534.222,374.565 \n",
       "  536.527,374.702 538.832,374.823 541.136,374.929 543.441,375.024 545.746,375.109 548.05,375.186 550.355,375.257 552.659,375.321 554.964,375.381 557.269,375.436 \n",
       "  559.573,375.487 561.878,375.534 564.183,375.578 566.487,375.619 568.792,375.657 571.096,375.693 573.401,375.726 575.706,375.757 578.01,375.786 580.315,375.813 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "471.37,262.931 562.315,262.931 562.315,232.691 471.37,232.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  471.37,262.931 562.315,262.931 562.315,232.691 471.37,232.691 471.37,262.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  477.37,247.811 513.37,247.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 519.37, 252.311)\" x=\"519.37\" y=\"252.311\">ysmin</text>\n",
       "</g>\n",
       "</svg>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting Done\n",
      "True loglikelihood      = -10460.417741142115\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<img src=\"\" />"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated loglikelihood = -10749.983591579816\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "DensityEstimationML.NeuralDensityEstimator{1}(Session(Ptr{Void} @0x00007fd84c8535b0), <Tensor Group:1 shape=unknown dtype=Complex{Float64}>, <Tensor Group_2:1 shape=unknown dtype=Any>, <Tensor t:1 shape=(1, ?) dtype=Float32>, <Tensor pdf:1 shape=unknown dtype=Float64>)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "est=demo(GenerateDatasets.Likas1(), [64, 64, 256], 10_000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "256-element Array{Float32,1}:\n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " 0.0       \n",
       " ⋮         \n",
       " 0.00465561\n",
       " 0.00490478\n",
       " 0.0061383 \n",
       " 0.00745164\n",
       " 0.00871045\n",
       " 0.014302  \n",
       " 0.0214234 \n",
       " 0.0640988 \n",
       " 0.21974   \n",
       " 0.390394  \n",
       " 0.620357  \n",
       " 4.23067   "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# serious levels of sparsity only a few components matter at all\n",
    "sort(component_weights(est))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1m\u001b[33mWARNING: \u001b[39m\u001b[22m\u001b[33mabs{T <: Number}(x::AbstractArray{T}) is deprecated, use abs.(x) instead.\u001b[39m\n",
      "Stacktrace:\n",
      " [1] \u001b[1mdepwarn\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::Symbol\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m./deprecated.jl:70\u001b[22m\u001b[22m\n",
      " [2] \u001b[1mabs\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Float32,1}\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m./deprecated.jl:57\u001b[22m\u001b[22m\n",
      " [3] \u001b[1m|>\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Array{Float32,1}, ::Base.#abs\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m./operators.jl:862\u001b[22m\u001b[22m\n",
      " [4] \u001b[1minclude_string\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::String\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m./loading.jl:515\u001b[22m\u001b[22m\n",
      " [5] \u001b[1minclude_string\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Module, ::String, ::String\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/home/uniwa/students2/students/20361362/linux/.julia/v0.6/Compat/src/Compat.jl:407\u001b[22m\u001b[22m\n",
      " [6] \u001b[1mexecute_request\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket, ::IJulia.Msg\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/home/uniwa/students2/students/20361362/linux/.julia/v0.6/IJulia/src/execute_request.jl:154\u001b[22m\u001b[22m\n",
      " [7] \u001b[1meventloop\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::ZMQ.Socket\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m/home/uniwa/students2/students/20361362/linux/.julia/v0.6/IJulia/src/eventloop.jl:8\u001b[22m\u001b[22m\n",
      " [8] \u001b[1m(::IJulia.##14#17)\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m./task.jl:335\u001b[22m\u001b[22m\n",
      "while loading In[15], in expression starting on line 1\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<img src=\"\" />"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check that sparsity\n",
    "run(est.sess, est.sess.graph[\"W_3\"]) |> vec |> abs |> histogram"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"600\" height=\"400\" viewBox=\"0 0 600 400\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip00\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"600\" height=\"400\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "0,400 600,400 600,0 0,0 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip01\">\n",
       "    <rect x=\"120\" y=\"0\" width=\"421\" height=\"400\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "55.6177,175.813 283.776,175.813 283.776,11.811 55.6177,11.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip02\">\n",
       "    <rect x=\"55\" y=\"11\" width=\"229\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  112.932,173.353 112.932,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  170.258,173.353 170.258,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  227.585,173.353 227.585,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,120.854 280.353,120.854 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,52.1781 280.353,52.1781 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,175.813 283.776,175.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  112.932,175.813 112.932,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  170.258,175.813 170.258,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  227.585,175.813 227.585,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,175.813 55.6177,11.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,120.854 59.04,120.854 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,52.1781 59.04,52.1781 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 112.932, 189.613)\" x=\"112.932\" y=\"189.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 170.258, 189.613)\" x=\"170.258\" y=\"189.613\">10000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 227.585, 189.613)\" x=\"227.585\" y=\"189.613\">15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 125.354)\" x=\"49.6177\" y=\"125.354\">10</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 56.6781)\" x=\"49.6177\" y=\"56.6781\">20</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,175.813 56.7642,158.02 57.9107,98.9957 59.0572,18.601 60.2038,11.811 61.3503,26.4063 62.4968,36.0311 63.6433,41.5054 64.7898,44.6288 65.9364,46.405 \n",
       "  67.0829,48.3374 68.2294,50.8328 69.3759,53.8728 70.5225,57.2002 71.669,60.4755 72.8155,63.4844 73.962,66.1998 75.1085,68.7129 76.2551,71.1554 77.4016,73.6598 \n",
       "  78.5481,76.3447 79.6946,79.3019 80.8412,82.52 81.9877,85.8124 83.1342,88.9435 84.2807,91.7851 85.4273,94.3434 86.5738,96.6414 87.7203,98.7154 88.8668,100.656 \n",
       "  90.0133,102.583 91.1599,104.589 92.3064,106.661 93.4529,108.735 94.5994,110.743 95.746,112.642 96.8925,114.41 98.039,116.037 99.1855,117.596 100.332,119.215 \n",
       "  101.479,120.881 102.625,122.512 103.772,124.067 104.918,125.52 106.065,126.877 107.211,128.145 108.358,129.322 109.504,130.459 110.651,131.538 111.797,132.585 \n",
       "  112.944,133.602 114.09,134.584 115.237,135.539 116.383,136.448 117.53,137.281 118.676,137.984 119.823,138.571 120.969,139.155 122.116,139.829 123.263,140.595 \n",
       "  124.409,141.413 125.556,142.236 126.702,143.02 127.849,143.707 128.995,143.665 130.142,143.605 131.288,144.124 132.435,144.743 133.581,145.399 134.728,146.066 \n",
       "  135.874,146.76 137.021,147.492 138.167,148.259 139.314,148.948 140.46,149.434 141.607,149.867 142.753,150.327 143.9,150.752 145.046,151.1 146.193,151.436 \n",
       "  147.339,151.844 148.486,152.353 149.633,152.93 150.779,153.545 151.926,154.179 153.072,154.82 154.219,155.453 155.365,156.065 156.512,156.645 157.658,157.186 \n",
       "  158.805,157.689 159.951,158.158 161.098,158.606 162.244,159.03 163.391,159.442 164.537,159.847 165.684,160.239 166.83,160.622 167.977,160.99 169.123,161.345 \n",
       "  170.27,161.685 171.416,162.006 172.563,162.31 173.71,162.595 174.856,162.87 176.003,163.125 177.149,163.371 178.296,163.599 179.442,163.818 180.589,164.028 \n",
       "  181.735,164.234 182.882,164.428 184.028,164.618 185.175,164.801 186.321,164.977 187.468,165.15 188.614,165.319 189.761,165.485 190.907,165.645 192.054,165.801 \n",
       "  193.2,165.954 194.347,166.107 195.493,166.254 196.64,166.4 197.786,166.544 198.933,166.702 200.08,166.886 201.226,167.056 202.373,167.21 203.519,167.342 \n",
       "  204.666,167.465 205.812,167.579 206.959,167.685 208.105,167.788 209.252,167.877 210.398,167.962 211.545,168.033 212.691,168.093 213.838,168.134 214.984,168.158 \n",
       "  216.131,168.161 217.277,168.146 218.424,168.11 219.57,168.053 220.717,168.043 221.863,167.936 223.01,165.11 224.157,165.123 225.303,165.498 226.45,165.874 \n",
       "  227.596,166.249 228.743,166.557 229.889,166.809 231.036,166.991 232.182,167.075 233.329,167.051 234.475,166.928 235.622,166.73 236.768,166.458 237.915,166.127 \n",
       "  239.061,165.747 240.208,165.325 241.354,164.866 242.501,164.381 243.647,163.853 244.794,163.308 245.94,162.734 247.087,162.117 248.233,161.459 249.38,160.779 \n",
       "  250.527,160.054 251.673,159.295 252.82,158.503 253.966,157.675 255.113,156.833 256.259,155.959 257.406,155.05 258.552,154.112 259.699,153.134 260.845,152.14 \n",
       "  261.992,151.09 263.138,150.015 264.285,148.9 265.431,147.743 266.578,146.557 267.724,145.36 268.871,144.119 270.017,142.829 271.164,141.486 272.31,140.158 \n",
       "  273.457,138.778 274.604,137.359 275.75,135.917 276.897,134.451 278.043,132.937 279.19,131.432 280.336,129.913 281.483,128.353 282.629,126.775 283.776,125.167 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "171.536,62.931 265.776,62.931 265.776,32.691 171.536,32.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  171.536,62.931 265.776,62.931 265.776,32.691 171.536,32.691 171.536,62.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  177.536,47.811 213.536,47.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 219.536, 52.311)\" x=\"219.536\" y=\"52.311\">ysmax</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "352.157,175.813 580.315,175.813 580.315,11.811 352.157,11.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip03\">\n",
       "    <rect x=\"352\" y=\"11\" width=\"229\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  409.472,173.353 409.472,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  466.798,173.353 466.798,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  524.124,173.353 524.124,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,146.519 576.893,146.519 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,95.2037 576.893,95.2037 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,43.8884 576.893,43.8884 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,175.813 580.315,175.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  409.472,175.813 409.472,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  466.798,175.813 466.798,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  524.124,175.813 524.124,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,175.813 352.157,11.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,146.519 355.579,146.519 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,95.2037 355.579,95.2037 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,43.8884 355.579,43.8884 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 409.472, 189.613)\" x=\"409.472\" y=\"189.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 466.798, 189.613)\" x=\"466.798\" y=\"189.613\">10000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 524.124, 189.613)\" x=\"524.124\" y=\"189.613\">15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 151.019)\" x=\"346.157\" y=\"151.019\">10000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 99.7037)\" x=\"346.157\" y=\"99.7037\">15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 48.3884)\" x=\"346.157\" y=\"48.3884\">20000</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,11.811 353.303,111.626 354.45,149.478 355.596,158.742 356.743,161.422 357.89,162.934 359.036,163.933 360.183,164.673 361.329,165.343 362.476,165.956 \n",
       "  363.622,166.513 364.769,167.024 365.915,167.499 367.062,167.944 368.208,168.366 369.355,168.765 370.501,169.143 371.648,169.5 372.794,169.836 373.941,170.153 \n",
       "  375.087,170.458 376.234,170.759 377.38,171.056 378.527,171.343 379.673,171.61 380.82,171.859 381.967,172.092 383.113,172.308 384.26,172.508 385.406,172.69 \n",
       "  386.553,172.854 387.699,172.998 388.846,173.124 389.992,173.231 391.139,173.323 392.285,173.403 393.432,173.475 394.578,173.553 395.725,173.639 396.871,173.716 \n",
       "  398.018,173.78 399.164,173.835 400.311,173.884 401.457,173.929 402.604,173.971 403.75,174.01 404.897,174.047 406.043,174.08 407.19,174.112 408.337,174.142 \n",
       "  409.483,174.17 410.63,174.197 411.776,174.223 412.923,174.248 414.069,174.272 415.216,174.296 416.362,174.321 417.509,174.345 418.655,174.368 419.802,174.389 \n",
       "  420.948,174.409 422.095,174.428 423.241,174.447 424.388,174.466 425.534,174.511 426.681,174.554 427.827,174.591 428.974,174.627 430.12,174.662 431.267,174.698 \n",
       "  432.414,174.732 433.56,174.765 434.707,174.795 435.853,174.822 437,174.847 438.146,174.871 439.293,174.894 440.439,174.917 441.586,174.94 442.732,174.965 \n",
       "  443.879,174.989 445.025,175.014 446.172,175.039 447.318,175.065 448.465,175.089 449.611,175.112 450.758,175.133 451.904,175.152 453.051,175.169 454.197,175.185 \n",
       "  455.344,175.2 456.49,175.214 457.637,175.228 458.784,175.24 459.93,175.252 461.077,175.264 462.223,175.275 463.37,175.286 464.516,175.296 465.663,175.306 \n",
       "  466.809,175.316 467.956,175.326 469.102,175.335 470.249,175.344 471.395,175.354 472.542,175.363 473.688,175.371 474.835,175.38 475.981,175.389 477.128,175.397 \n",
       "  478.274,175.406 479.421,175.414 480.567,175.422 481.714,175.43 482.861,175.437 484.007,175.445 485.154,175.453 486.3,175.46 487.447,175.467 488.593,175.474 \n",
       "  489.74,175.481 490.886,175.488 492.033,175.494 493.179,175.501 494.326,175.507 495.472,175.513 496.619,175.519 497.765,175.525 498.912,175.531 500.058,175.537 \n",
       "  501.205,175.542 502.351,175.547 503.498,175.552 504.644,175.557 505.791,175.562 506.938,175.566 508.084,175.57 509.231,175.574 510.377,175.578 511.524,175.582 \n",
       "  512.67,175.585 513.817,175.588 514.963,175.591 516.11,175.594 517.256,175.597 518.403,175.606 519.549,175.616 520.696,175.624 521.842,175.633 522.989,175.642 \n",
       "  524.135,175.651 525.282,175.66 526.428,175.668 527.575,175.676 528.721,175.683 529.868,175.69 531.014,175.696 532.161,175.703 533.308,175.708 534.454,175.714 \n",
       "  535.601,175.719 536.747,175.724 537.894,175.729 539.04,175.733 540.187,175.738 541.333,175.742 542.48,175.746 543.626,175.75 544.773,175.753 545.919,175.757 \n",
       "  547.066,175.76 548.212,175.763 549.359,175.766 550.505,175.769 551.652,175.772 552.798,175.775 553.945,175.777 555.091,175.779 556.238,175.782 557.385,175.784 \n",
       "  558.531,175.786 559.678,175.788 560.824,175.79 561.971,175.792 563.117,175.794 564.264,175.795 565.41,175.797 566.557,175.799 567.703,175.8 568.85,175.802 \n",
       "  569.996,175.803 571.143,175.804 572.289,175.805 573.436,175.807 574.582,175.808 575.729,175.809 576.875,175.81 578.022,175.811 579.168,175.812 580.315,175.813 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "433.823,62.931 562.315,62.931 562.315,32.691 433.823,32.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  433.823,62.931 562.315,62.931 562.315,32.691 433.823,32.691 433.823,62.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  439.823,47.811 475.823,47.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 481.823, 52.311)\" x=\"481.823\" y=\"52.311\">working_loss</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "55.6177,375.813 283.776,375.813 283.776,211.811 55.6177,211.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip04\">\n",
       "    <rect x=\"55\" y=\"211\" width=\"229\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  112.932,373.353 112.932,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  170.258,373.353 170.258,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  227.585,373.353 227.585,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,343.736 280.353,343.736 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,292.421 280.353,292.421 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  59.04,241.105 280.353,241.105 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,375.813 283.776,375.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  112.932,375.813 112.932,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  170.258,375.813 170.258,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  227.585,375.813 227.585,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,375.813 55.6177,211.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,343.736 59.04,343.736 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,292.421 59.04,292.421 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,241.105 59.04,241.105 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 112.932, 389.613)\" x=\"112.932\" y=\"389.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 170.258, 389.613)\" x=\"170.258\" y=\"389.613\">10000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 227.585, 389.613)\" x=\"227.585\" y=\"389.613\">15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 348.236)\" x=\"49.6177\" y=\"348.236\">-20000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 296.921)\" x=\"49.6177\" y=\"296.921\">-15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 49.6177, 245.605)\" x=\"49.6177\" y=\"245.605\">-10000</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  55.6177,375.813 56.7642,275.994 57.9107,238.038 59.0572,228.405 60.2038,225.684 61.3503,224.265 62.4968,223.321 63.6433,222.61 64.7898,221.956 65.9364,221.35 \n",
       "  67.0829,220.802 68.2294,220.301 69.3759,219.84 70.5225,219.408 71.669,219.001 72.8155,218.614 73.962,218.247 75.1085,217.9 76.2551,217.573 77.4016,217.265 \n",
       "  78.5481,216.97 79.6946,216.68 80.8412,216.395 81.9877,216.12 83.1342,215.863 84.2807,215.624 85.4273,215.399 86.5738,215.189 87.7203,214.996 88.8668,214.82 \n",
       "  90.0133,214.661 91.1599,214.523 92.3064,214.403 93.4529,214.302 94.5994,214.215 95.746,214.14 96.8925,214.072 98.039,213.999 99.1855,213.915 100.332,213.842 \n",
       "  101.479,213.782 102.625,213.731 103.772,213.685 104.918,213.643 106.065,213.604 107.211,213.567 108.358,213.533 109.504,213.502 110.651,213.472 111.797,213.444 \n",
       "  112.944,213.417 114.09,213.392 115.237,213.368 116.383,213.345 117.53,213.322 118.676,213.299 119.823,213.275 120.969,213.252 122.116,213.23 123.263,213.21 \n",
       "  124.409,213.191 125.556,213.173 126.702,213.155 127.849,213.137 128.995,213.092 130.142,213.049 131.288,213.013 132.435,212.978 133.581,212.943 134.728,212.908 \n",
       "  135.874,212.874 137.021,212.843 138.167,212.814 139.314,212.787 140.46,212.762 141.607,212.739 142.753,212.717 143.9,212.694 145.046,212.671 146.193,212.647 \n",
       "  147.339,212.623 148.486,212.599 149.633,212.574 150.779,212.549 151.926,212.526 153.072,212.503 154.219,212.483 155.365,212.464 156.512,212.447 157.658,212.432 \n",
       "  158.805,212.417 159.951,212.403 161.098,212.39 162.244,212.378 163.391,212.366 164.537,212.355 165.684,212.344 166.83,212.334 167.977,212.324 169.123,212.314 \n",
       "  170.27,212.304 171.416,212.295 172.563,212.286 173.71,212.276 174.856,212.267 176.003,212.259 177.149,212.25 178.296,212.241 179.442,212.233 180.589,212.224 \n",
       "  181.735,212.216 182.882,212.208 184.028,212.2 185.175,212.192 186.321,212.185 187.468,212.177 188.614,212.17 189.761,212.163 190.907,212.156 192.054,212.149 \n",
       "  193.2,212.142 194.347,212.135 195.493,212.129 196.64,212.122 197.786,212.116 198.933,212.11 200.08,212.104 201.226,212.098 202.373,212.092 203.519,212.087 \n",
       "  204.666,212.081 205.812,212.076 206.959,212.071 208.105,212.067 209.252,212.062 210.398,212.058 211.545,212.054 212.691,212.05 213.838,212.046 214.984,212.042 \n",
       "  216.131,212.039 217.277,212.036 218.424,212.033 219.57,212.03 220.717,212.028 221.863,212.019 223.01,212.008 224.157,211.999 225.303,211.991 226.45,211.982 \n",
       "  227.596,211.973 228.743,211.964 229.889,211.956 231.036,211.948 232.182,211.941 233.329,211.934 234.475,211.928 235.622,211.922 236.768,211.916 237.915,211.91 \n",
       "  239.061,211.905 240.208,211.9 241.354,211.896 242.501,211.891 243.647,211.887 244.794,211.883 245.94,211.879 247.087,211.875 248.233,211.871 249.38,211.868 \n",
       "  250.527,211.864 251.673,211.861 252.82,211.858 253.966,211.855 255.113,211.853 256.259,211.85 257.406,211.847 258.552,211.845 259.699,211.843 260.845,211.84 \n",
       "  261.992,211.838 263.138,211.836 264.285,211.834 265.431,211.832 266.578,211.831 267.724,211.829 268.871,211.827 270.017,211.826 271.164,211.824 272.31,211.823 \n",
       "  273.457,211.822 274.604,211.82 275.75,211.819 276.897,211.818 278.043,211.817 279.19,211.816 280.336,211.814 281.483,211.813 282.629,211.812 283.776,211.811 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "140.567,262.931 265.776,262.931 265.776,232.691 140.567,232.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  140.567,262.931 265.776,262.931 265.776,232.691 140.567,232.691 140.567,262.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  146.567,247.811 182.567,247.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 188.567, 252.311)\" x=\"188.567\" y=\"252.311\">loglikelihood</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "352.157,375.813 580.315,375.813 580.315,211.811 352.157,211.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip05\">\n",
       "    <rect x=\"352\" y=\"211\" width=\"229\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  409.472,373.353 409.472,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  466.798,373.353 466.798,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  524.124,373.353 524.124,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,343.045 576.893,343.045 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,287.271 576.893,287.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  355.579,231.496 576.893,231.496 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,375.813 580.315,375.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  409.472,375.813 409.472,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  466.798,375.813 466.798,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  524.124,375.813 524.124,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,375.813 352.157,211.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,343.045 355.579,343.045 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,287.271 355.579,287.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,231.496 355.579,231.496 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 409.472, 389.613)\" x=\"409.472\" y=\"389.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 466.798, 389.613)\" x=\"466.798\" y=\"389.613\">10000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 524.124, 389.613)\" x=\"524.124\" y=\"389.613\">15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 347.545)\" x=\"346.157\" y=\"347.545\">2.5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 291.771)\" x=\"346.157\" y=\"291.771\">5.0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 346.157, 235.996)\" x=\"346.157\" y=\"235.996\">7.5</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  352.157,375.181 353.303,368.546 354.45,355.769 355.596,346.71 356.743,345.316 357.89,345.743 359.036,346.314 360.183,347.065 361.329,348.002 362.476,349.076 \n",
       "  363.622,350.279 364.769,351.571 365.915,352.902 367.062,354.225 368.208,355.509 369.355,356.735 370.501,357.895 371.648,358.989 372.794,360.022 373.941,360.997 \n",
       "  375.087,361.922 376.234,362.807 377.38,363.656 378.527,364.461 379.673,365.21 380.82,365.897 381.967,366.524 383.113,367.105 384.26,367.649 385.406,368.158 \n",
       "  386.553,368.628 387.699,369.056 388.846,369.442 389.992,369.791 391.139,370.111 392.285,370.407 393.432,370.682 394.578,370.933 395.725,371.151 396.871,371.336 \n",
       "  398.018,371.516 399.164,371.703 400.311,371.892 401.457,372.076 402.604,372.252 403.75,372.419 404.897,372.579 406.043,372.733 407.19,372.882 408.337,373.024 \n",
       "  409.483,373.159 410.63,373.285 411.776,373.402 412.923,373.508 414.069,373.604 415.216,373.691 416.362,373.772 417.509,373.85 418.655,373.921 419.802,373.981 \n",
       "  420.948,374.028 422.095,374.059 423.241,374.076 424.388,374.087 425.534,374.12 426.681,374.237 427.827,374.38 428.974,374.523 430.12,374.654 431.267,374.771 \n",
       "  432.414,374.875 433.56,374.966 434.707,375.043 435.853,375.104 437,375.149 438.146,375.191 439.293,375.23 440.439,375.267 441.586,375.305 442.732,375.348 \n",
       "  443.879,375.391 445.025,375.436 446.172,375.482 447.318,375.528 448.465,375.572 449.611,375.612 450.758,375.649 451.904,375.681 453.051,375.709 454.197,375.734 \n",
       "  455.344,375.755 456.49,375.773 457.637,375.787 458.784,375.798 459.93,375.807 461.077,375.812 462.223,375.813 463.37,375.812 464.516,375.807 465.663,375.798 \n",
       "  466.809,375.785 467.956,375.768 469.102,375.746 470.249,375.72 471.395,375.688 472.542,375.65 473.688,375.607 474.835,375.556 475.981,375.499 477.128,375.434 \n",
       "  478.274,375.36 479.421,375.278 480.567,375.186 481.714,375.083 482.861,374.969 484.007,374.843 485.154,374.704 486.3,374.55 487.447,374.381 488.593,374.194 \n",
       "  489.74,373.989 490.886,373.763 492.033,373.515 493.179,373.242 494.326,372.943 495.472,372.614 496.619,372.251 497.765,371.848 498.912,371.404 500.058,370.918 \n",
       "  501.205,370.392 502.351,369.826 503.498,369.218 504.644,368.569 505.791,367.876 506.938,367.144 508.084,366.37 509.231,365.558 510.377,364.705 511.524,363.814 \n",
       "  512.67,362.885 513.817,361.919 514.963,360.919 516.11,359.9 517.256,359.045 518.403,360.055 519.549,359.086 520.696,359.397 521.842,359.705 522.989,359.768 \n",
       "  524.135,359.583 525.282,359.139 526.428,358.485 527.575,357.675 528.721,356.69 529.868,355.545 531.014,354.265 532.161,352.883 533.308,351.394 534.454,349.8 \n",
       "  535.601,348.119 536.747,346.357 537.894,344.524 539.04,342.635 540.187,340.625 541.333,338.595 542.48,336.486 543.626,334.237 544.773,331.886 545.919,329.479 \n",
       "  547.066,326.943 548.212,324.304 549.359,321.58 550.505,318.75 551.652,315.887 552.798,312.929 553.945,309.857 555.091,306.709 556.238,303.436 557.385,300.109 \n",
       "  558.531,296.63 559.678,293.062 560.824,289.377 561.971,285.561 563.117,281.658 564.264,277.722 565.41,273.64 566.557,269.41 567.703,265.013 568.85,260.67 \n",
       "  569.996,256.156 571.143,251.532 572.289,246.828 573.436,242.041 574.582,237.116 575.729,232.222 576.875,227.274 578.022,222.188 579.168,217.051 580.315,211.811 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "471.37,262.931 562.315,262.931 562.315,232.691 471.37,232.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  471.37,262.931 562.315,262.931 562.315,232.691 471.37,232.691 471.37,262.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  477.37,247.811 513.37,247.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 519.37, 252.311)\" x=\"519.37\" y=\"252.311\">ysmin</text>\n",
       "</g>\n",
       "</svg>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting Done\n",
      "True loglikelihood      = -7128.92733457648\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<img src=\"\" />"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated loglikelihood = -7145.545686143989\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "DensityEstimationML.NeuralDensityEstimator{1}(Session(Ptr{Void} @0x00007fd84c460e90), <Tensor Group:1 shape=unknown dtype=Any>, <Tensor Group_2:1 shape=unknown dtype=Any>, <Tensor t:1 shape=(1, ?) dtype=Float32>, <Tensor pdf:1 shape=unknown dtype=Float64>)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demo(GenerateDatasets.Likas2(), [64,64], 20_000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"600\" height=\"400\" viewBox=\"0 0 600 400\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip00\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"600\" height=\"400\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "0,400 600,400 600,0 0,0 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip01\">\n",
       "    <rect x=\"120\" y=\"0\" width=\"421\" height=\"400\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "42.4384,175.813 282.128,175.813 282.128,11.811 42.4384,11.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip02\">\n",
       "    <rect x=\"42\" y=\"11\" width=\"241\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  102.942,173.353 102.942,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  163.47,173.353 163.47,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  223.997,173.353 223.997,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  46.0338,93.9144 278.533,93.9144 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,175.813 282.128,175.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  102.942,175.813 102.942,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  163.47,175.813 163.47,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  223.997,175.813 223.997,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,175.813 42.4384,11.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,93.9144 46.0338,93.9144 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 102.942, 189.613)\" x=\"102.942\" y=\"189.613\">2500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 163.47, 189.613)\" x=\"163.47\" y=\"189.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 223.997, 189.613)\" x=\"223.997\" y=\"189.613\">7500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 36.4384, 98.4144)\" x=\"36.4384\" y=\"98.4144\">1.5</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,11.811 44.8595,141.942 47.2806,160.575 49.7017,165.211 52.1228,165.432 54.544,159.046 56.9651,136.174 59.3862,95.6025 61.8073,54.4866 64.2284,29.3565 \n",
       "  66.6495,19.7531 69.0706,19.7417 71.4917,25.563 73.9128,35.2789 76.3339,47.6984 78.7551,61.909 81.1762,77.1225 83.5973,92.6323 86.0184,107.807 88.4395,122.1 \n",
       "  90.8606,135.055 93.2817,146.325 95.7028,155.674 98.1239,162.995 100.545,168.334 102.966,171.568 105.387,172.989 107.808,173.781 110.229,174.295 112.651,174.641 \n",
       "  115.072,174.877 117.493,175.029 119.914,175.167 122.335,175.25 124.756,175.336 127.177,175.389 129.598,175.446 132.019,175.482 134.441,175.523 136.862,175.548 \n",
       "  139.283,175.579 141.704,175.597 144.125,175.621 146.546,175.634 148.967,175.652 151.388,175.661 153.809,175.676 156.231,175.687 158.652,175.697 161.073,175.709 \n",
       "  163.494,175.714 165.915,175.724 168.336,175.728 170.757,175.737 173.178,175.741 175.599,175.749 178.021,175.751 180.442,175.758 182.863,175.758 185.284,175.764 \n",
       "  187.705,175.771 190.126,175.771 192.547,175.776 194.968,175.775 197.389,175.779 199.811,175.784 202.232,175.783 204.653,175.787 207.074,175.791 209.495,175.79 \n",
       "  211.916,175.794 214.337,175.794 216.758,175.796 219.179,175.8 221.601,175.798 224.022,175.802 226.443,175.801 228.864,175.803 231.285,175.806 233.706,175.803 \n",
       "  236.127,175.806 238.548,175.809 240.969,175.807 243.391,175.81 245.812,175.807 248.233,175.809 250.654,175.812 253.075,175.807 255.496,175.809 257.917,175.811 \n",
       "  260.338,175.812 262.759,175.811 265.181,175.813 267.602,175.809 270.023,175.811 272.444,175.813 274.865,175.811 277.286,175.811 279.707,175.813 282.128,175.811 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "169.888,62.931 264.128,62.931 264.128,32.691 169.888,32.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  169.888,62.931 264.128,62.931 264.128,32.691 169.888,32.691 169.888,62.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  175.888,47.811 211.888,47.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 217.888, 52.311)\" x=\"217.888\" y=\"52.311\">ysmax</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "340.625,175.813 580.315,175.813 580.315,11.811 340.625,11.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip03\">\n",
       "    <rect x=\"340\" y=\"11\" width=\"241\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  401.129,173.353 401.129,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  461.656,173.353 461.656,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  522.184,173.353 522.184,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  344.22,133.778 576.72,133.778 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  344.22,90.1172 576.72,90.1172 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  344.22,46.4564 576.72,46.4564 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,175.813 580.315,175.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  401.129,175.813 401.129,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  461.656,175.813 461.656,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  522.184,175.813 522.184,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,175.813 340.625,11.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,133.778 344.22,133.778 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,90.1172 344.22,90.1172 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,46.4564 344.22,46.4564 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 401.129, 189.613)\" x=\"401.129\" y=\"189.613\">2500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 461.656, 189.613)\" x=\"461.656\" y=\"189.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 522.184, 189.613)\" x=\"522.184\" y=\"189.613\">7500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 334.625, 138.278)\" x=\"334.625\" y=\"138.278\">825</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 334.625, 94.6172)\" x=\"334.625\" y=\"94.6172\">850</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 334.625, 50.9564)\" x=\"334.625\" y=\"50.9564\">875</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,11.811 343.046,32.0806 345.467,33.2096 347.888,33.4347 350.31,33.725 352.731,41.424 355.152,52.2245 357.573,60.9785 359.994,67.9877 362.415,73.7318 \n",
       "  364.836,78.622 367.257,82.8786 369.678,86.6313 372.099,89.9698 374.521,92.9635 376.942,95.6672 379.363,98.1264 381.784,100.38 384.205,102.458 386.626,104.389 \n",
       "  389.047,106.194 391.468,107.89 393.889,109.491 396.311,111.007 398.732,112.446 401.153,113.814 403.574,115.12 405.995,116.384 408.416,117.619 410.837,118.834 \n",
       "  413.258,120.037 415.679,121.229 418.101,122.413 420.522,123.59 422.943,124.759 425.364,125.92 427.785,127.072 430.206,128.213 432.627,129.343 435.048,130.461 \n",
       "  437.469,131.568 439.891,132.661 442.312,133.741 444.733,134.808 447.154,135.861 449.575,136.9 451.996,137.925 454.417,138.935 456.838,139.934 459.259,140.915 \n",
       "  461.681,141.886 464.102,142.841 466.523,143.783 468.944,144.711 471.365,145.626 473.786,146.528 476.207,147.417 478.628,148.292 481.049,149.155 483.471,150.005 \n",
       "  485.892,150.844 488.313,151.67 490.734,152.484 493.155,153.287 495.576,154.078 497.997,154.857 500.418,155.626 502.839,156.384 505.261,157.131 507.682,157.868 \n",
       "  510.103,158.594 512.524,159.306 514.945,160.016 517.366,160.712 519.787,161.399 522.208,162.076 524.629,162.741 527.051,163.402 529.472,164.052 531.893,164.693 \n",
       "  534.314,165.324 536.735,165.946 539.156,166.563 541.577,167.169 543.998,167.767 546.419,168.357 548.841,168.939 551.262,169.513 553.683,170.079 556.104,170.637 \n",
       "  558.525,171.179 560.946,171.732 563.367,172.267 565.788,172.795 568.209,173.316 570.631,173.83 573.052,174.336 575.473,174.836 577.894,175.329 580.315,175.813 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "433.823,62.931 562.315,62.931 562.315,32.691 433.823,32.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  433.823,62.931 562.315,62.931 562.315,32.691 433.823,32.691 433.823,62.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  439.823,47.811 475.823,47.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 481.823, 52.311)\" x=\"481.823\" y=\"52.311\">working_loss</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "42.4384,375.813 282.128,375.813 282.128,211.811 42.4384,211.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip04\">\n",
       "    <rect x=\"42\" y=\"211\" width=\"241\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  102.942,373.353 102.942,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  163.47,373.353 163.47,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  223.997,373.353 223.997,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  46.0338,341.205 278.533,341.205 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  46.0338,297.59 278.533,297.59 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  46.0338,253.976 278.533,253.976 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,375.813 282.128,375.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  102.942,375.813 102.942,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  163.47,375.813 163.47,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  223.997,375.813 223.997,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,375.813 42.4384,211.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,341.205 46.0338,341.205 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,297.59 46.0338,297.59 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,253.976 46.0338,253.976 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 102.942, 389.613)\" x=\"102.942\" y=\"389.613\">2500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 163.47, 389.613)\" x=\"163.47\" y=\"389.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 223.997, 389.613)\" x=\"223.997\" y=\"389.613\">7500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 36.4384, 345.705)\" x=\"36.4384\" y=\"345.705\">-875</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 36.4384, 302.09)\" x=\"36.4384\" y=\"302.09\">-850</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 36.4384, 258.476)\" x=\"36.4384\" y=\"258.476\">-825</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  42.4384,375.813 44.8595,355.454 47.2806,354.294 49.7017,354.06 52.1228,353.769 54.544,346.09 56.9651,335.339 59.3862,326.647 61.8073,319.68 64.2284,313.954 \n",
       "  66.6495,309.072 69.0706,304.82 71.4917,301.07 73.9128,297.732 76.3339,294.736 78.7551,292.027 81.1762,289.559 83.5973,287.294 86.0184,285.2 88.4395,283.252 \n",
       "  90.8606,281.43 93.2817,279.718 95.7028,278.102 98.1239,276.574 100.545,275.126 102.966,273.753 105.387,272.446 107.808,271.181 110.229,269.947 112.651,268.733 \n",
       "  115.072,267.53 117.493,266.339 119.914,265.156 122.335,263.98 124.756,262.812 127.177,261.653 129.598,260.502 132.019,259.362 134.441,258.233 136.862,257.116 \n",
       "  139.283,256.01 141.704,254.918 144.125,253.839 146.546,252.773 148.967,251.721 151.388,250.684 153.809,249.66 156.231,248.651 158.652,247.653 161.073,246.673 \n",
       "  163.494,245.703 165.915,244.748 168.336,243.807 170.757,242.88 173.178,241.966 175.599,241.066 178.021,240.178 180.442,239.304 182.863,238.442 185.284,237.592 \n",
       "  187.705,236.754 190.126,235.929 192.547,235.116 194.968,234.314 197.389,233.524 199.811,232.745 202.232,231.977 204.653,231.22 207.074,230.474 209.495,229.738 \n",
       "  211.916,229.013 214.337,228.301 216.758,227.592 219.179,226.896 221.601,226.211 224.022,225.534 226.443,224.87 228.864,224.209 231.285,223.56 233.706,222.92 \n",
       "  236.127,222.289 238.548,221.668 240.969,221.052 243.391,220.447 245.812,219.849 248.233,219.26 250.654,218.678 253.075,218.105 255.496,217.539 257.917,216.982 \n",
       "  260.338,216.44 262.759,215.889 265.181,215.354 267.602,214.827 270.023,214.306 272.444,213.792 274.865,213.287 277.286,212.788 279.707,212.295 282.128,211.811 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "138.919,262.931 264.128,262.931 264.128,232.691 138.919,232.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  138.919,262.931 264.128,262.931 264.128,232.691 138.919,232.691 138.919,262.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  144.919,247.811 180.919,247.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 186.919, 252.311)\" x=\"186.919\" y=\"252.311\">loglikelihood</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "340.625,375.813 580.315,375.813 580.315,211.811 340.625,211.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip05\">\n",
       "    <rect x=\"340\" y=\"211\" width=\"241\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  401.129,373.353 401.129,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  461.656,373.353 461.656,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  522.184,373.353 522.184,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  344.22,345.085 576.72,345.085 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  344.22,314.356 576.72,314.356 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  344.22,283.626 576.72,283.626 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  344.22,252.896 576.72,252.896 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  344.22,222.167 576.72,222.167 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,375.813 580.315,375.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  401.129,375.813 401.129,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  461.656,375.813 461.656,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  522.184,375.813 522.184,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,375.813 340.625,211.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,345.085 344.22,345.085 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,314.356 344.22,314.356 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,283.626 344.22,283.626 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,252.896 344.22,252.896 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,222.167 344.22,222.167 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 401.129, 389.613)\" x=\"401.129\" y=\"389.613\">2500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 461.656, 389.613)\" x=\"461.656\" y=\"389.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 522.184, 389.613)\" x=\"522.184\" y=\"389.613\">7500</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 334.625, 349.585)\" x=\"334.625\" y=\"349.585\">1.01</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 334.625, 318.856)\" x=\"334.625\" y=\"318.856\">1.02</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 334.625, 288.126)\" x=\"334.625\" y=\"288.126\">1.03</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 334.625, 257.396)\" x=\"334.625\" y=\"257.396\">1.04</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 334.625, 226.667)\" x=\"334.625\" y=\"226.667\">1.05</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  340.625,352.211 343.046,357.499 345.467,366.579 347.888,369.009 350.31,368.709 352.731,357.884 355.152,327.533 357.573,279.862 359.994,236.457 362.415,214.512 \n",
       "  364.836,211.811 367.257,220.397 369.678,234.926 372.099,252.342 374.521,270.73 376.942,288.805 379.363,305.705 381.784,320.89 384.205,334.066 386.626,345.131 \n",
       "  389.047,354.116 391.468,361.153 393.889,366.434 396.311,370.197 398.732,372.708 401.153,374.121 403.574,374.733 405.995,375.062 408.416,375.268 410.837,375.402 \n",
       "  413.258,375.492 415.679,375.551 418.101,375.601 420.522,375.632 422.943,375.662 425.364,375.681 427.785,375.7 430.206,375.713 432.627,375.726 435.048,375.735 \n",
       "  437.469,375.744 439.891,375.751 442.312,375.758 444.733,375.763 447.154,375.768 449.575,375.771 451.996,375.775 454.417,375.778 456.838,375.782 459.259,375.785 \n",
       "  461.681,375.787 464.102,375.789 466.523,375.791 468.944,375.793 471.365,375.794 473.786,375.796 476.207,375.797 478.628,375.798 481.049,375.799 483.471,375.8 \n",
       "  485.892,375.801 488.313,375.802 490.734,375.803 493.155,375.804 495.576,375.804 497.997,375.805 500.418,375.806 502.839,375.806 505.261,375.807 507.682,375.807 \n",
       "  510.103,375.808 512.524,375.808 514.945,375.809 517.366,375.809 519.787,375.809 522.208,375.81 524.629,375.81 527.051,375.81 529.472,375.81 531.893,375.811 \n",
       "  534.314,375.811 536.735,375.811 539.156,375.811 541.577,375.812 543.998,375.812 546.419,375.812 548.841,375.812 551.262,375.812 553.683,375.812 556.104,375.812 \n",
       "  558.525,375.813 560.946,375.813 563.367,375.813 565.788,375.813 568.209,375.813 570.631,375.813 573.052,375.813 575.473,375.813 577.894,375.813 580.315,375.813 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "471.37,262.931 562.315,262.931 562.315,232.691 471.37,232.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  471.37,262.931 562.315,262.931 562.315,232.691 471.37,232.691 471.37,262.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  477.37,247.811 513.37,247.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 519.37, 252.311)\" x=\"519.37\" y=\"252.311\">ysmin</text>\n",
       "</g>\n",
       "</svg>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting Done\n",
      "True loglikelihood      = -789.8449596782973\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<img src=\"\" />"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated loglikelihood = -800.5568562850015\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "DensityEstimationML.NeuralDensityEstimator{1}(Session(Ptr{Void} @0x00007fd84cae2970), <Tensor Group:1 shape=unknown dtype=Any>, <Tensor Group_2:1 shape=unknown dtype=Any>, <Tensor t:1 shape=(1, ?) dtype=Float32>, <Tensor pdf:1 shape=unknown dtype=Float64>)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demo(GenerateDatasets.MagdonIsmailAndAtiya(), [32], 10_000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"600\" height=\"400\" viewBox=\"0 0 600 400\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip00\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"600\" height=\"400\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "0,400 600,400 600,0 0,0 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip01\">\n",
       "    <rect x=\"120\" y=\"0\" width=\"421\" height=\"400\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "49.028,175.813 283.776,175.813 283.776,11.811 49.028,11.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip02\">\n",
       "    <rect x=\"49\" y=\"11\" width=\"236\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  107.998,173.353 107.998,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  166.98,173.353 166.98,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  225.962,173.353 225.962,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  52.5492,137.063 280.254,137.063 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  52.5492,72.6388 280.254,72.6388 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,175.813 283.776,175.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  107.998,175.813 107.998,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  166.98,175.813 166.98,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  225.962,175.813 225.962,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,175.813 49.028,11.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,137.063 52.5492,137.063 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,72.6388 52.5492,72.6388 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 107.998, 189.613)\" x=\"107.998\" y=\"189.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 166.98, 189.613)\" x=\"166.98\" y=\"189.613\">10000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 225.962, 189.613)\" x=\"225.962\" y=\"189.613\">15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 43.028, 141.563)\" x=\"43.028\" y=\"141.563\">5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 43.028, 77.1388)\" x=\"43.028\" y=\"77.1388\">10</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip02)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,175.813 50.2077,159.298 51.3873,156.866 52.5669,156.996 53.7466,155.556 54.9262,152.513 56.1059,146.887 57.2855,136.437 58.4651,116.272 59.6448,80.4274 \n",
       "  60.8244,47.2085 62.004,38.7205 63.1837,38.23 64.3633,38.9872 65.5429,38.9397 66.7226,33.702 67.9022,22.4275 69.0819,16.4021 70.2615,13.2396 71.4411,11.811 \n",
       "  72.6208,12.0378 73.8004,13.4789 74.98,15.5941 76.1597,17.85 77.3393,19.2824 78.5189,21.1338 79.6986,23.9263 80.8782,26.86 82.0579,30.1623 83.2375,33.7294 \n",
       "  84.4171,36.4287 85.5968,39.0798 86.7764,42.7622 87.956,46.9123 89.1357,50.5035 90.3153,53.6082 91.4949,56.6846 92.6746,59.8133 93.8542,62.8283 95.0339,65.6914 \n",
       "  96.2135,68.429 97.3931,71.0459 98.5728,73.5427 99.7524,75.9212 100.932,78.1793 102.112,80.3108 103.291,82.3045 104.471,84.1517 105.651,85.8421 106.83,87.376 \n",
       "  108.01,88.762 109.189,90.0137 110.369,91.1413 111.549,92.153 112.728,93.0519 113.908,93.8254 115.088,94.4251 116.267,95.1291 117.447,95.8733 118.627,96.6254 \n",
       "  119.806,97.6752 120.986,98.8778 122.165,100.129 123.345,101.405 124.525,102.679 125.704,103.964 126.884,105.255 128.064,106.547 129.243,107.831 130.423,109.109 \n",
       "  131.603,110.374 132.782,111.624 133.962,112.836 135.141,114.022 136.321,115.17 137.501,116.286 138.68,117.363 139.86,118.39 141.04,119.382 142.219,120.326 \n",
       "  143.399,121.232 144.579,122.099 145.758,122.888 146.938,123.638 148.118,124.355 149.297,125.048 150.477,125.719 151.656,126.379 152.836,127.016 154.016,127.617 \n",
       "  155.195,128.195 156.375,128.736 157.555,129.252 158.734,129.721 159.914,130.147 161.094,130.531 162.273,130.87 163.453,131.155 164.632,131.405 165.812,131.615 \n",
       "  166.992,131.252 168.171,131.23 169.351,131.403 170.531,131.638 171.71,131.972 172.89,132.393 174.07,132.85 175.249,133.3 176.429,133.725 177.608,134.126 \n",
       "  178.788,134.488 179.968,134.816 181.147,135.112 182.327,135.384 183.507,135.644 184.686,135.896 185.866,136.131 187.046,136.364 188.225,136.586 189.405,136.815 \n",
       "  190.584,137.04 191.764,137.265 192.944,137.493 194.123,137.722 195.303,137.959 196.483,138.191 197.662,138.429 198.842,138.661 200.022,138.905 201.201,139.144 \n",
       "  202.381,139.381 203.56,139.625 204.74,139.87 205.92,140.113 207.099,140.366 208.279,140.605 209.459,140.872 210.638,141.129 211.818,141.403 212.998,141.675 \n",
       "  214.177,141.965 215.357,142.252 216.536,142.563 217.716,142.845 218.896,143.166 220.075,143.459 221.255,143.757 222.435,144.058 223.614,144.337 224.794,144.631 \n",
       "  225.974,144.93 227.153,145.261 228.333,145.599 229.512,145.964 230.692,146.345 231.872,146.74 233.051,147.152 234.231,147.564 235.411,147.984 236.59,148.401 \n",
       "  237.77,148.815 238.95,149.223 240.129,149.626 241.309,150.019 242.488,150.41 243.668,150.793 244.848,151.154 246.027,151.508 247.207,151.858 248.387,152.192 \n",
       "  249.566,152.508 250.746,152.815 251.926,153.12 253.105,153.414 254.285,153.697 255.464,153.971 256.644,154.233 257.824,154.493 259.003,154.728 260.183,154.971 \n",
       "  261.363,155.195 262.542,155.42 263.722,155.639 264.902,155.85 266.081,156.054 267.261,156.25 268.44,156.445 269.62,156.626 270.8,156.806 271.979,156.979 \n",
       "  273.159,157.148 274.339,157.309 275.518,157.468 276.698,157.624 277.878,157.777 279.057,157.928 280.237,158.071 281.416,158.208 282.596,158.344 283.776,158.475 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "171.536,62.931 265.776,62.931 265.776,32.691 171.536,32.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  171.536,62.931 265.776,62.931 265.776,32.691 171.536,32.691 171.536,62.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  177.536,47.811 213.536,47.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 219.536, 52.311)\" x=\"219.536\" y=\"52.311\">ysmax</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "345.567,175.813 580.315,175.813 580.315,11.811 345.567,11.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip03\">\n",
       "    <rect x=\"345\" y=\"11\" width=\"236\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  404.537,173.353 404.537,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  463.519,173.353 463.519,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  522.501,173.353 522.501,14.2711 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  349.089,158.338 576.794,158.338 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  349.089,96.2665 576.794,96.2665 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  349.089,34.1953 576.794,34.1953 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,175.813 580.315,175.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  404.537,175.813 404.537,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  463.519,175.813 463.519,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  522.501,175.813 522.501,173.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,175.813 345.567,11.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,158.338 349.089,158.338 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,96.2665 349.089,96.2665 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,34.1953 349.089,34.1953 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 404.537, 189.613)\" x=\"404.537\" y=\"189.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 463.519, 189.613)\" x=\"463.519\" y=\"189.613\">10000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 522.501, 189.613)\" x=\"522.501\" y=\"189.613\">15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 339.567, 162.838)\" x=\"339.567\" y=\"162.838\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 339.567, 100.766)\" x=\"339.567\" y=\"100.766\">6000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 339.567, 38.6953)\" x=\"339.567\" y=\"38.6953\">7000</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip03)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,11.811 346.747,72.0074 347.927,76.9869 349.106,77.634 350.286,78.3931 351.465,79.2907 352.645,80.3434 353.825,81.5698 355.004,82.9764 356.184,84.4694 \n",
       "  357.364,85.7743 358.543,86.8921 359.723,87.9048 360.903,88.85 362.082,89.8462 363.262,91.2826 364.441,93.208 365.621,95.0991 366.801,97.0079 367.98,98.8325 \n",
       "  369.16,100.456 370.34,101.871 371.519,103.114 372.699,104.241 373.879,105.342 375.058,106.491 376.238,107.693 377.417,108.888 378.597,110.046 379.777,111.194 \n",
       "  380.956,112.33 382.136,113.538 383.316,114.849 384.495,116.168 385.675,117.398 386.855,118.57 388.034,119.731 389.214,120.872 390.393,121.981 391.573,123.062 \n",
       "  392.753,124.118 393.932,125.154 395.112,126.167 396.292,127.155 397.471,128.116 398.651,129.048 399.831,129.953 401.01,130.831 402.19,131.686 403.369,132.521 \n",
       "  404.549,133.338 405.729,134.138 406.908,134.923 408.088,135.693 409.268,136.45 410.447,137.199 411.627,137.973 412.807,138.744 413.986,139.5 415.166,140.227 \n",
       "  416.345,140.928 417.525,141.609 418.705,142.272 419.884,142.916 421.064,143.542 422.244,144.15 423.423,144.742 424.603,145.318 425.783,145.88 426.962,146.428 \n",
       "  428.142,146.964 429.321,147.487 430.501,147.999 431.681,148.505 432.86,149.002 434.04,149.49 435.22,149.968 436.399,150.437 437.579,150.895 438.759,151.341 \n",
       "  439.938,151.778 441.118,152.202 442.297,152.613 443.477,153.014 444.657,153.404 445.836,153.783 447.016,154.153 448.196,154.514 449.375,154.866 450.555,155.213 \n",
       "  451.735,155.551 452.914,155.883 454.094,156.21 455.273,156.532 456.453,156.849 457.633,157.161 458.812,157.47 459.992,157.775 461.172,158.077 462.351,158.376 \n",
       "  463.531,158.68 464.711,158.99 465.89,159.302 467.07,159.611 468.249,159.913 469.429,160.212 470.609,160.506 471.788,160.798 472.968,161.086 474.148,161.373 \n",
       "  475.327,161.657 476.507,161.938 477.687,162.218 478.866,162.495 480.046,162.769 481.225,163.041 482.405,163.31 483.585,163.575 484.764,163.837 485.944,164.096 \n",
       "  487.124,164.352 488.303,164.603 489.483,164.851 490.663,165.095 491.842,165.335 493.022,165.572 494.201,165.804 495.381,166.033 496.561,166.258 497.74,166.48 \n",
       "  498.92,166.698 500.1,166.912 501.279,167.123 502.459,167.331 503.639,167.536 504.818,167.736 505.998,167.936 507.177,168.131 508.357,168.323 509.537,168.512 \n",
       "  510.716,168.699 511.896,168.882 513.076,169.062 514.255,169.24 515.435,169.413 516.615,169.588 517.794,169.758 518.974,169.925 520.153,170.091 521.333,170.254 \n",
       "  522.513,170.413 523.692,170.573 524.872,170.729 526.052,170.882 527.231,171.033 528.411,171.182 529.591,171.329 530.77,171.473 531.95,171.617 533.129,171.757 \n",
       "  534.309,171.896 535.489,172.032 536.668,172.167 537.848,172.298 539.028,172.429 540.207,172.556 541.387,172.683 542.567,172.806 543.746,172.928 544.926,173.048 \n",
       "  546.106,173.164 547.285,173.279 548.465,173.393 549.644,173.504 550.824,173.613 552.004,173.721 553.183,173.826 554.363,173.929 555.543,174.031 556.722,174.131 \n",
       "  557.902,174.228 559.082,174.325 560.261,174.42 561.441,174.513 562.62,174.604 563.8,174.694 564.98,174.782 566.159,174.87 567.339,174.956 568.519,175.04 \n",
       "  569.698,175.123 570.878,175.204 572.058,175.284 573.237,175.364 574.417,175.442 575.596,175.518 576.776,175.594 577.956,175.668 579.135,175.741 580.315,175.813 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "433.823,62.931 562.315,62.931 562.315,32.691 433.823,32.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  433.823,62.931 562.315,62.931 562.315,32.691 433.823,32.691 433.823,62.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  439.823,47.811 475.823,47.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 481.823, 52.311)\" x=\"481.823\" y=\"52.311\">working_loss</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "49.028,375.813 283.776,375.813 283.776,211.811 49.028,211.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip04\">\n",
       "    <rect x=\"49\" y=\"211\" width=\"236\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  107.998,373.353 107.998,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  166.98,373.353 166.98,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  225.962,373.353 225.962,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  52.5492,353.431 280.254,353.431 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  52.5492,291.364 280.254,291.364 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  52.5492,229.297 280.254,229.297 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,375.813 283.776,375.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  107.998,375.813 107.998,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  166.98,375.813 166.98,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  225.962,375.813 225.962,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,375.813 49.028,211.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,353.431 52.5492,353.431 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,291.364 52.5492,291.364 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,229.297 52.5492,229.297 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 107.998, 389.613)\" x=\"107.998\" y=\"389.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 166.98, 389.613)\" x=\"166.98\" y=\"389.613\">10000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 225.962, 389.613)\" x=\"225.962\" y=\"389.613\">15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 43.028, 357.931)\" x=\"43.028\" y=\"357.931\">-7000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 43.028, 295.864)\" x=\"43.028\" y=\"295.864\">-6000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 43.028, 233.797)\" x=\"43.028\" y=\"233.797\">-5000</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip04)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  49.028,375.813 50.2077,315.62 51.3873,310.641 52.5669,309.993 53.7466,309.233 54.9262,308.333 56.1059,307.274 57.2855,306.031 58.4651,304.573 59.6448,302.923 \n",
       "  60.8244,301.385 62.004,300.178 63.1837,299.142 64.3633,298.187 65.5429,297.177 66.7226,295.679 67.9022,293.627 69.0819,291.659 70.2615,289.704 71.4411,287.853 \n",
       "  72.6208,286.224 73.8004,284.821 74.98,283.597 76.1597,282.493 77.3393,281.405 78.5189,280.274 79.6986,279.1 80.8782,277.935 82.0579,276.81 83.2375,275.698 \n",
       "  84.4171,274.588 85.5968,273.406 86.7764,272.131 87.956,270.85 89.1357,269.654 90.3153,268.51 91.4949,267.375 92.6746,266.261 93.8542,265.177 95.0339,264.12 \n",
       "  96.2135,263.086 97.3931,262.071 98.5728,261.077 99.7524,260.107 100.932,259.162 102.112,258.245 103.291,257.355 104.471,256.489 105.651,255.646 106.83,254.821 \n",
       "  108.01,254.013 109.189,253.22 110.369,252.443 111.549,251.679 112.728,250.928 113.908,250.184 115.088,249.413 116.267,248.647 117.447,247.895 118.627,247.173 \n",
       "  119.806,246.478 120.986,245.804 122.165,245.148 123.345,244.511 124.525,243.892 125.704,243.291 126.884,242.706 128.064,242.137 129.243,241.582 130.423,241.04 \n",
       "  131.603,240.51 132.782,239.993 133.962,239.487 135.141,238.987 136.321,238.495 137.501,238.012 138.68,237.538 139.86,237.074 141.04,236.62 142.219,236.178 \n",
       "  143.399,235.745 144.579,235.325 145.758,234.917 146.938,234.519 148.118,234.132 149.297,233.755 150.477,233.388 151.656,233.029 152.836,232.679 154.016,232.335 \n",
       "  155.195,231.999 156.375,231.669 157.555,231.344 158.734,231.023 159.914,230.708 161.094,230.397 162.273,230.089 163.453,229.785 164.632,229.484 165.812,229.186 \n",
       "  166.992,228.881 168.171,228.571 169.351,228.259 170.531,227.951 171.71,227.65 172.89,227.353 174.07,227.06 175.249,226.77 176.429,226.483 177.608,226.197 \n",
       "  178.788,225.914 179.968,225.634 181.147,225.355 182.327,225.079 183.507,224.805 184.686,224.535 185.866,224.266 187.046,224.002 188.225,223.74 189.405,223.482 \n",
       "  190.584,223.227 191.764,222.976 192.944,222.729 194.123,222.486 195.303,222.246 196.483,222.011 197.662,221.779 198.842,221.55 200.022,221.326 201.201,221.105 \n",
       "  202.381,220.888 203.56,220.674 204.74,220.464 205.92,220.257 207.099,220.053 208.279,219.853 209.459,219.654 210.638,219.46 211.818,219.269 212.998,219.08 \n",
       "  214.177,218.894 215.357,218.711 216.536,218.532 217.716,218.355 218.896,218.182 220.075,218.008 221.255,217.839 222.435,217.672 223.614,217.508 224.794,217.346 \n",
       "  225.974,217.187 227.153,217.028 228.333,216.873 229.512,216.72 230.692,216.57 231.872,216.422 233.051,216.276 234.231,216.133 235.411,215.99 236.59,215.85 \n",
       "  237.77,215.713 238.95,215.577 240.129,215.443 241.309,215.312 242.488,215.183 243.668,215.056 244.848,214.93 246.027,214.807 247.207,214.686 248.387,214.567 \n",
       "  249.566,214.451 250.746,214.337 251.926,214.224 253.105,214.113 254.285,214.004 255.464,213.897 256.644,213.792 257.824,213.689 259.003,213.588 260.183,213.489 \n",
       "  261.363,213.391 262.542,213.295 263.722,213.201 264.902,213.108 266.081,213.017 267.261,212.927 268.44,212.84 269.62,212.752 270.8,212.666 271.979,212.582 \n",
       "  273.159,212.5 274.339,212.419 275.518,212.339 276.698,212.26 277.878,212.182 279.057,212.105 280.237,212.03 281.416,211.956 282.596,211.883 283.776,211.811 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "140.567,262.931 265.776,262.931 265.776,232.691 140.567,232.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  140.567,262.931 265.776,262.931 265.776,232.691 140.567,232.691 140.567,262.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  146.567,247.811 182.567,247.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 188.567, 252.311)\" x=\"188.567\" y=\"252.311\">loglikelihood</text>\n",
       "</g>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "345.567,375.813 580.315,375.813 580.315,211.811 345.567,211.811 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip05\">\n",
       "    <rect x=\"345\" y=\"211\" width=\"236\" height=\"165\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  404.537,373.353 404.537,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  463.519,373.353 463.519,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  522.501,373.353 522.501,214.271 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  349.089,311.143 576.794,311.143 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#000000; stroke-width:0.5; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  349.089,246.441 576.794,246.441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,375.813 580.315,375.813 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  404.537,375.813 404.537,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  463.519,375.813 463.519,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  522.501,375.813 522.501,373.353 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,375.813 345.567,211.811 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,311.143 349.089,311.143 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,246.441 349.089,246.441 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 404.537, 389.613)\" x=\"404.537\" y=\"389.613\">5000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 463.519, 389.613)\" x=\"463.519\" y=\"389.613\">10000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:middle;\" transform=\"rotate(0, 522.501, 389.613)\" x=\"522.501\" y=\"389.613\">15000</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 339.567, 315.643)\" x=\"339.567\" y=\"315.643\">1.5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:end;\" transform=\"rotate(0, 339.567, 250.941)\" x=\"339.567\" y=\"250.941\">2.0</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip05)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  345.567,363.005 346.747,273.327 347.927,249.715 349.106,259.235 350.286,264.769 351.465,267.396 352.645,266.108 353.825,258.932 355.004,242.879 356.184,219.148 \n",
       "  357.364,211.811 358.543,228.856 359.723,249.019 360.903,266.824 362.082,281.57 363.262,292.622 364.441,300.564 365.621,309.404 366.801,318.495 367.98,326.362 \n",
       "  369.16,332.857 370.34,338.205 371.519,342.614 372.699,346.256 373.879,349.175 375.058,351.973 376.238,354.858 377.417,357.484 378.597,359.813 379.777,361.934 \n",
       "  380.956,363.737 382.136,365.304 383.316,366.846 384.495,368.245 385.675,369.37 386.855,370.284 388.034,371.075 389.214,371.764 390.393,372.337 391.573,372.805 \n",
       "  392.753,373.192 393.932,373.518 395.112,373.793 396.292,374.026 397.471,374.222 398.651,374.387 399.831,374.525 401.01,374.638 402.19,374.729 403.369,374.802 \n",
       "  404.549,374.858 405.729,374.9 406.908,374.93 408.088,374.95 409.268,374.963 410.447,374.974 411.627,375.003 412.807,375.067 413.986,375.127 415.166,375.178 \n",
       "  416.345,375.226 417.525,375.27 418.705,375.309 419.884,375.344 421.064,375.376 422.244,375.404 423.423,375.43 424.603,375.454 425.783,375.476 426.962,375.496 \n",
       "  428.142,375.515 429.321,375.532 430.501,375.547 431.681,375.562 432.86,375.577 434.04,375.59 435.22,375.602 436.399,375.612 437.579,375.622 438.759,375.63 \n",
       "  439.938,375.637 441.118,375.643 442.297,375.649 443.477,375.654 444.657,375.658 445.836,375.663 447.016,375.667 448.196,375.671 449.375,375.675 450.555,375.679 \n",
       "  451.735,375.683 452.914,375.687 454.094,375.691 455.273,375.694 456.453,375.698 457.633,375.701 458.812,375.704 459.992,375.707 461.172,375.71 462.351,375.712 \n",
       "  463.531,375.712 464.711,375.717 465.89,375.723 467.07,375.73 468.249,375.736 469.429,375.741 470.609,375.745 471.788,375.749 472.968,375.752 474.148,375.755 \n",
       "  475.327,375.758 476.507,375.76 477.687,375.761 478.866,375.763 480.046,375.764 481.225,375.765 482.405,375.765 483.585,375.766 484.764,375.766 485.944,375.766 \n",
       "  487.124,375.766 488.303,375.765 489.483,375.765 490.663,375.765 491.842,375.764 493.022,375.763 494.201,375.763 495.381,375.762 496.561,375.761 497.74,375.76 \n",
       "  498.92,375.76 500.1,375.759 501.279,375.758 502.459,375.758 503.639,375.757 504.818,375.757 505.998,375.757 507.177,375.757 508.357,375.757 509.537,375.757 \n",
       "  510.716,375.757 511.896,375.758 513.076,375.758 514.255,375.759 515.435,375.76 516.615,375.76 517.794,375.761 518.974,375.761 520.153,375.761 521.333,375.762 \n",
       "  522.513,375.762 523.692,375.763 524.872,375.764 526.052,375.765 527.231,375.767 528.411,375.768 529.591,375.77 530.77,375.772 531.95,375.774 533.129,375.775 \n",
       "  534.309,375.777 535.489,375.779 536.668,375.781 537.848,375.782 539.028,375.784 540.207,375.786 541.387,375.787 542.567,375.788 543.746,375.79 544.926,375.791 \n",
       "  546.106,375.793 547.285,375.794 548.465,375.795 549.644,375.796 550.824,375.797 552.004,375.798 553.183,375.799 554.363,375.8 555.543,375.801 556.722,375.802 \n",
       "  557.902,375.803 559.082,375.804 560.261,375.805 561.441,375.805 562.62,375.806 563.8,375.807 564.98,375.807 566.159,375.808 567.339,375.808 568.519,375.809 \n",
       "  569.698,375.81 570.878,375.81 572.058,375.811 573.237,375.811 574.417,375.811 575.596,375.812 576.776,375.812 577.956,375.813 579.135,375.813 580.315,375.813 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip00)\" points=\"\n",
       "471.37,262.931 562.315,262.931 562.315,232.691 471.37,232.691 \n",
       "  \" fill=\"#ffffff\" fill-opacity=\"1\"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#000000; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  471.37,262.931 562.315,262.931 562.315,232.691 471.37,232.691 471.37,262.931 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip00)\" style=\"stroke:#009af9; stroke-width:1; stroke-opacity:1; fill:none\" points=\"\n",
       "  477.37,247.811 513.37,247.811 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip00)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:12; text-anchor:start;\" transform=\"rotate(0, 519.37, 252.311)\" x=\"519.37\" y=\"252.311\">ysmin</text>\n",
       "</g>\n",
       "</svg>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting Done\n",
      "True loglikelihood      = -4308.792670836306\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<img src=\"\" />"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated loglikelihood = -4717.131383487122\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "DensityEstimationML.NeuralDensityEstimator{1}(Session(Ptr{Void} @0x00007fd84c3fb1f0), <Tensor Group:1 shape=unknown dtype=Any>, <Tensor Group_2:1 shape=unknown dtype=Any>, <Tensor t:1 shape=(1, ?) dtype=Float32>, <Tensor pdf:1 shape=unknown dtype=Float64>)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demo(Arcsine(1,4), [64,64], 20_000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WIP >1D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-09-22 20:10:33.089528: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:01:00.0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pdf = <Tensor pdf:1 shape=unknown dtype=Float64>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "DensityEstimationML.NeuralDensityEstimator{2}(Session(Ptr{Void} @0x00007fd84d33df30), <Tensor Group:1 shape=unknown dtype=Float32>, <Tensor Group_2:1 shape=unknown dtype=Any>, <Tensor t:1 shape=(2, ?) dtype=Float32>, <Tensor pdf:1 shape=unknown dtype=Float64>)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset = GenerateDatasets.Likas3()\n",
    "data = original_sample(dataset)\n",
    "est = NeuralDensityEstimator([64], approximate_support(GenerateDatasets.Likas3()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "@show loglikelihood(dataset, data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "est = NeuralDensityEstimator([64], approximate_support(GenerateDatasets.Likas3()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "condition!(est)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\n",
    "fit!(est, data'; epochs=10_000, callback=running_plot())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "\n",
    "fit!(est, data; epochs=10_000, callback=running_plot())\n",
    "println(\"Fitting Done\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sess = est.sess\n",
    "run(sess, est.pdf, Dict(est.t=>[0.1 0.1; 0.1 0.1; 0.3 0.6]'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sess = est.sess\n",
    "run(sess, sess.graph[\"numerator\"], Dict(est.t=>[0.1 0.1; 0.1 0.1; 0.3 0.6]'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "run(sess, gather(constant(data), 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "plotly()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "    meshgrid(xs, ys, zfun)\n",
    "\n",
    "Evalates `zfun(x,y)` at each point in `xs`, and `ys`.\n",
    "Returns 3 vectors, a list of x points a list of y points and the value of z at that point.\n",
    "\"\"\"\n",
    "function meshgrid(xs, ys, zfun) \n",
    "    # There is a cute generalisation of this with a `@generated` function\n",
    "    xpoints = eltype(xs)[]\n",
    "    ypoints = eltype(ys)[]\n",
    "    zpoints = typeof(zfun(xs[1],ys[1]))[]\n",
    "    sizehint!.([xpoints, ypoints, zpoints], length(xs)*length(ys))\n",
    "    for x in xs, y in ys\n",
    "        push!(xpoints,x)\n",
    "        push!(ypoints,y)\n",
    "        push!(zpoints,zfun(x,y))\n",
    "    end\n",
    "    \n",
    "    xpoints, ypoints, zpoints\n",
    "end\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "X=-0.1:0.01:0.3\n",
    "Y=-0.1:0.01:0.3\n",
    "scatter3d(meshgrid(X,Y, (x,y)->pdf(est, [x,y]))...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pdf(est, SMatrix{1,2}([0.1 0.1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "est"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "size([0.1 0.1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?SArray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 0.6.0",
   "language": "julia",
   "name": "julia-0.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
